ASTRID2 Error Fix
Status:
This documentation applies to version 1.50.1009.85 (14. January-2026 of the VdAstrid2/CVdAst2ErrorFix device.
Contents:
- CVdAst2ErrorFix overview
- Error Fix device
- ASTRID2 error fix scheduler device
- ASTRID2 automation device
- Vd2Astrid
- F0: A2 ring error recovery
- F1: A2 Transfer beamline error recovery
- F2: Very poor A2 beam lifetime which does not go away after an injection
- F3: A2 Post filling, trig auto tune correction
CVdAst2ErrorFix overview
Device design notes - see document(s) in folder
O:\Nat_ISA-Ast2Doc\machine\Documentation\Controls\Automation\ErrorFix
This devices implements error fix routines for ASTRID2.
F0: A2 ring error recovery
An errorFix to recover operation if there is trouble with BMH100IPS, QMH101IPS, QMH102IPS or any of the A2 IFA corrector supply distributors.
Cluster name: EFX01A2RINGERast2
Trigger
MachineMode: StatusAst2.MachineMode in OnAutoRecovery mode and
Trig if one of the supplies 'Main' is off
BMH100IPSast2.Main
QMH101IPSast2.Main
QMH102IPSast2.Main
Trig if SumFailure in any of the A2 distributors
Trig if Beam loss > 80 mA
Trig if Mains brownout and Beam loss > 10 mA
Missing documentation
Action
Set Decaying via Scheduler
Wait 30 s
Test that ConSys connections are OK (no network failure)
Wait up to 15 minutes for ConSys connections to be OK
Check that MachineMode is still in OnAutoRecovery - if not so, abort action.
Trigger a AutoOnOffAst2.turnRingOn (which turns "everything" on, inclusive a magnet wash).
Wait for AutoOnOffAst2.DeviceBusy going false
If AutoOnOffAst2.lstActionSuccess is false, then trigger another try to turn ring on
and wait for AutoOnOffAst2.DeviceBusy going false
Wait 30 seconds
If AutoOnOffAst2.lstActionSuccess Go To Filling (or Top-up) via Scheduler
Enable Trig of F3, Post Filling error fix actions.
Action Progress
0: Inactive
Trigged - actionProgress is OR between
1: BMH101 Main off
2: QMH101 Main off
4: QMH102 Main off
8: One of the A2 Corrector distributers has failure
Recover try in progress (n is try number - 1,2):
n00xx: Wait (xx seconds left). *1
n1xxx: Check connections (xxx seconds waited). *1
n5000: 'Turn Ring On' send
n6xxx: Waiting for 'deviceBusy' FALSE (xxx seconds waited). *1
n9000: Timeout while waiting for 'deviceBusy' FALSE
n9001: ConSys connection failed (invalid value(s)
n9002: Recovery aborted - machineMode no longer in OnAutoRecovery
Recovered:
8xx: Wait after recovered (xx seconds left). *1
F1: A2 Transfer beamline error recovery
An errorFix to recover operation need an ErrorFix to recover operation if there is trouble with power supplies in the transfer Beamline.
Cluster name: EFX01A2TRNBLERast2
Trigger
MachineMode: StatusAst2.MachineMode in OnRecovery mode and
Trig if one of the supplies 'Main' is off:
SMH111IPUast2.Main
FMH161IPUast2.Main
FMH111IPUast2.Main
FMH121IPUast2.Main
QMV211IPSast2.Main
QMH211IPSast2.Main
QMH212IPSast2.Main
QMV212IPSast2.Main
BMV211IPSast2.Main
BMV212IPSast2.Main
If trigged, EFX01A2TRNBLERast2.triggeredBy is set to word with bits set for supplies having main failure (bit numbers in the order of the supplies listed above, i.e. SMH111IPUast2.Main =0x01,FMH161IPUast2.Main =0x02 etc.
Action
Test and wait for ConSys connections are OK (no network failure)
Wait additional 30 s
Check that MachineMode is still in OnAutoRecovery - if not so, abort action.
If A2ringErrorRecovery is not running (if running just exit because then the action is done in A2ringErrorRecovery)
Set Decaying via Scheduler
Read state of AutoOnOffAst2.WashWhenTurningOn
Set AutoOnOffAst2.WashWhenTurningOn false
Recover try in progress (n is try number - 1,2):
Check that MachineMode is still in OnAutoRecovery - if not so, abort action.
Trigger a AutoOnOffAst2.turnRingOn (which turns "everything" on, but without the magnet wash).
Wait for AutoOnOffAst2.DeviceBusy going false
Check AutoOnOffAst2.lstActionsuccess - if ok, recovered.
Set back the old state of AutoOnOffAst2.WashWhenTurningOn
If recovered
Wait 30 seconds
Go To Filling (or Top-up) via Scheduler
Action Progress
0: Inactive
1: Trigged - triggeredBy is OR between
0x01 SMH111IPUast2.Main off
0x02 FMH161IPUast2.Main off
0x04 FMH111IPUast2.Main off
0x08 FMH121IPUast2.Main off
0x10 QMV211IPSast2.Main off
0x20 QMH211IPSast2.Main off
0x40 QMH212IPSast2.Main off
0x80 QMV212IPSast2.Main off
0x100 BMV211IPSast2.Main off
0x200 BMV212IPSast2.Main off
2: Waiting for ConSys connections ok
1xx: Wait (xx seconds left). *1
1000: A2 Ring Recovery is active - no action needed - already being done by A2 Ring Recovery.
3: Recovery aborted, MachineMode is no longer in OnAutoRecovery.
4: Set Decaying
Recover try in progress (n is try number - 1,2):
n1000: 'Turn Ring On' send
n2xxx: Waiting for 'deviceBusy' FALSE (xxx seconds waited). *1
n3000: Waiting for 'deviceBusy' timeout
n4000: No success, not all mains on
n5000: No success, lstActionsuccess false
n6000: Recovery aborted - MachineMode no longer in OnAutoRecovery
n6001: Recovery aborted while waiting for turn ring on finisg - MachineMode no longer in OnAutoRecovery
n9000: Try n Turn ring on successful
8xx: Waiting for things to stabilize (xx seconds left). *1
5: Restore BeamMode
F2: Very poor A2 beam lifetime which does not go away after an injection
One time recover try of very poor A2 beam lifetime which does not go away after an injection (dust particle?).
Cluster name: EFX100A2POORLFTMast2
Trigger
ErrorFix is On
StatusAst2.MachineMode in OnRecovery mode
AND (1 <= A2BeamCurrent.normal <=20)
AND (A2BeamCurrent.tau20s < 0.1 h )
AND (InjectionAst2.Efficiency >= 25%)
Condition must be true for (at least) 1 minute (we need to have at least two injections - to verify transfer efficiency is good)
Action
Kick out beam (insert viewer MVS141MNPast2.ctrl) and refill (Scheduler Request Beam-mode restore).
This is a one time fix, as there could be other reasons for very poor A2 beam lifetime. State should go to 'failNoBlock' in case of failure.
Action Progress
0: Inactive
1..60: Trigger condition fulfilled count - action is trigged when the counter reaches 60
1000: Action trigged, Viewer In
2000: Viewer Out
3xxx: Waiting for scheduler restore beam mode (filling),xxx seconds waited
4xxx: Beam mode is filling - Waiting 3 minutes, xxx seconds left
5000: Recovered, filling ok
6000: Not recovered, Filling conditions not ok
7000: Not recovered, Beam mode not in filling
F3: A2 Post filling
The post filling error fix are when going to beam mode Top-up if other error fix methods has signalled that this fix should be run. In present implementation, F0: 'A2 ring error recovery' signals Post filling to be run when its actions are finished.
Post filling will issue an Auto tune correction two minutes after going to filling. The beam mode filling mode must occur within some minutes after the method is signalled.
Cluster name: EFX100A2POSTFILLast2
Trigger
ErrorFix is On
StatusAst2.MachineMode in OnRecovery mode
All of the following be fullfilled:
F3 signalled to run within last x (60 minutes)
Trigger waiting and beam mode goes to Top-up
Beamcurrent > AutomationAst2.TopUpISet
OR manual trigged
Action
Wait 1 minute
Do a A2 tune correction
Wait for the tune correction to finish
Wait 1 minute
If AutoAlignAstInj.State is Inactive then do an microtron auto align
Wait for mictron autoalign finished
Action Progress
0: Inactive
1: Trigged
2: Action running
1xxx: Wait (xxx seconds left). *1
2000: Set auto correct tunes, AutoOnOffAst2.CorrectTunes TRUE
2xxx: Waiting for 'correctTunes' FALSE (xxx seconds waited).*1
4000: Tune correction finished
4001: Wait for correctTunes !Busy timed out
5xxx: Wait (xxx seconds left). *1
6000: Set ASTRID injection auto adjust, AutoAlignAstInj.AlignAfterNext TRUE
6xxx: Waiting for 'AlignAfterNext' FALSE (xxx seconds waited).*1
8000: ASTRID injection auto adjust finished
8001: Wait for 'ASTRID injection auto adjust' !Busy timed out
*1
ActionProgress: When counting up/down, only the first and the last value are added to the read/write histories in order to make the histories more readable.
Last Modified 15 January 2026